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Abstract 


All  the  standard  methods  for  avoiding  cycling  in  the  simplex  algorithm  set  up  a 
subprogram  related  to  the  blocking  rows  in  the  canonical  form  and  look  for  an  improv¬ 
ing  homogeneous  solution.  The  method  proposed  in  this  paper  does  not.  Instead,  the 
Gass-Saaty  parametric  method  applied  to  the  objective  (cost)  form  is  used  to  choose 
the  incoming  column  [2).  The  pivot-row  choice  among  blocking  rows,  i.e.,  those  tied  for 
pivot,  is  arbitrary.  A  simple  anti-cycling  device  is  used  which  avoids  dual  degeneracy 
of  the  parameterized  objective  with  probability  one.  Tests  were  run  on  nine  highly 
degenerate  practical  test  problems  ranging  in  size  from  small  to  large.  Using  MINOS 
software  [5],  the  standard  simplex  method  required  31,195  iterations  of  which  20,504 
(or  66%)  were  blocked  pivots.  Using  the  parametric  scheme  required  14,379  iterations, 
of  which  9,481  or  66%  were  also  blocked.  The  reduction  in  the  number  of  iterations 
for  this  set  of  highly  degenerate  test  problems  is  over  50%.  CPU  time  reduction  under 
the  current  way  of  implementation  is  40%. 


1.  Introduction  /'T 

f  o'*'0 

The  linear  program  which  we  wish  to  solve  using  the  simplex  method  is  V  ^.tc 

\ 

FIND  min  z,  Ax  =  b,  cx  —  z,  x  >  0  . 

The  algorithm  is  said  to  stall  (or  be  blocked)  when  one  or  more  pivot  steps  results  in  no 
change  in  the  objective  z.  It  is  generally  believed  (but  this  turns  out  to  be  wrong)  that  ^T~ 
the  way  to  make  progress  during  a  stall  is  to  select  the  “right”  row  to  pivot  on  among  the 
blocking  rows  of  the  canonical  form  of  iteration  t.  Thus,  the  e-perturbation  as  implemented 
using  the  lexicographic  rule,  the  inductive  method  as  implemented  by  Wolfe’s  rule,  and 
the  random-row  choice  rule  are  all  examples  of  row-selection  rules  for  avoiding  cycling. 
Bland’s  rule  is  a  scheme  that  selects  both  the  column  and  the  row.  From  the  poi.it  of  view 
cf  reducing  the  number  of  iterations  during  a  stall,  little  good  can  be  said  about  any  of  T 


them. 
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The  method  presented  here  is  a  rule  for  selecting  the  incoming  column  only.  Once  the 
column  is  selected,  any  row  among  the  set  of  blocking  rows  of  the  canonical  form  may  be 
used  for  pivoting.  For  numerical  stability,  pivoting  on  the  largest  coefficient  among  the 
blocking  rows  is  recommended.  Except  for  column  selection,  all  other  steps  are  the  same 
as  the  standard  simplex  method.  We  will  therefore  not  review  these  and  will  assume  they 
are  known  to  the  reader. 


2.  Gass-Saaty  Parametric  Method 

The  rule  proposed  for  choosing  the  incoming  column  is  the  one  used  in  the  paramet¬ 
ric  method  of  Gass  and  Saaty  [2];  it  reduces  dual  infeasibility.  A  perturbation  of  the 
parametrized  objective  is  introduced  to  avoid  dual  degeneracy  with  probability  one.  We 
will  first  state  their  column-selection  scheme  in  the  framework  of  the  canonical  form.  We 
then  discuss  how  the  updates  can  be  easily  computed  from  the  data  of  the  original  problem 
using  the  framework  of  the  revised  simplex  method  and  LU  factorization  of  the  basis. 

We  assume  that  a  starting  feasible  basis  B°  is  given  and  0°  —  {jf, . . . .  j°n}  is  the  set 
of  basic  column  indices  j  of  the  initial  basis.  An  additional  equation  dx  =  w  is  used  to 
parameterize  the  cost  equation  cx  =  z.  Let 


y.  djXj  =  w  where 


(  dj  —  0 

1  4  =  \\A, 


for  j  e  0° 
11(1 +  <j)  for  j  i  0° 


where  £y  is  chosen  from  a  table  of  random  numbers  0  <  £y  <  1  and  ||A.y||  is  the  Euclidean 
norm,  some  other  norm,  or  any  other  arbitrarily  chosen  positive  number. 

Denote  by  B t_1  the  feasible  basis  at  the  start  of  iteration  t  and  denote  by  c p  and  dp 
the  subset  of  components  of  c  and  d  corresponding  to  basic  column  indices  0  =  0t_1.  Let 
7r  and  a  be  found  by  solving 

cp  =  7r Bt~l  and  dp  =  oB*~l  . 

On  iteration  t,  the  parameterized  reduced  cost  form  is 


where 


(c  +  9d)x  =  (z  —  Zt-i)  +  0(w  —  xvt-\) 


c  =  c  —  a  A  and  d  =  d  —  a  A  . 


If  c  >  0,  then  6  —  0,  and  the  iterative  process  STOPS  with  the  current  basic  feasible 
solution  optimal. 

Otherwise  at  least  one  cy  <  0.  We  make  the  inductive  assumption,  for  all  6  in  some 
range  0  <  6‘  <  6  <  6(~l,  that 

£j(0)  =  Cj  +  9dj  >  0  for  all  j  $  0  . 


Moreover,  we  make  the  uniqueness  assumption  that  at  0  =  0‘: 


Cj(6t)  =  Cj  +  0*dj  >  0  for  all  j  ^  0  except  j  =  s  , 

£•{&)  =  c,  +  0‘ d,  =  0  for  some  s  £  0  . 

The  choice  of  s  will  be  unique  with  probability  one  because  of  the  random  selection  of  (j. 

Note  that  the  choice  of  column  s  is  “unit  free”,  i.e.,  it  would  not  be  affected  if  all  columns 

A.j  were  rescaled  by  positive  factors  when  dj  is  initially  chosen  proportional  to  some  norm 
of  A.j. 

It  is  easy  to  see  that  d,  >  0  because  for  6*  <  0  <  0‘-1  we  have  c,  +  0d ,  >  0;  subtracting 
c,  +  =  0  yields  (0  —  0*)d,  >  0  where  ( 0  -  0‘)  >  0.  Moreover,  c,  =  —0ld,  <  0.  We  will 

make  use  of  the  fact  that  d,  >  0  and  c,  <  0. 

According  to  the  theory  of  Gass  and  Saaty  [2],  see  also  [l],  after  a  pivot  in  the  selected 
column  s,  it  is  possible  once  again  to  decrease  0  by  a  positive  amount.  The  proof  is  as 
follows:  After  pivoting  on  some  row  r,  for  j  ^  jr,  j  ^  0  <  0l: 

Cj{0)  =  ( Cj  -  arjc,/ar,)  +  0(dj  -  arj_d,/ar,)_ 

=  ( Cj  -  arjC,/ars)  +  {0  -0‘){dj  -  arid,/ar,)  +  0*{dj  -  arjd,/ar.) 

=  (dj  +  0ldj)  +  (0  —  0*)(dj  —  arjd,/ar,)  , 

where  we  have  dropped  the  second  term  from  the  first  and  third  parenthesis  because 
c,  +  O'd,  =  0,  by  the  definition  of  0l .  Note  Cj  +  0ldj  >  0  by  the  uniqueness  assumption 
for  j  f3\j  s.  Therefore,  for  some  range  0  <  0l  for  j  &  /3*~l  and  j  /  s,Cj{0 )  >  0.  For 
3  ~  jr,  we  have  for  all  0  <  0l: 

Cjr(0)  =  -(c,  +  0d,)JaT, 

=  {0-  0‘)(-d,/6r,)  >  0 

because  ar,  >  0  and  d„  >  0  as  we  have  shown  earlier.  Let  f3l  be  the  updated  set  of  basic 
indices.  Since  now  (cy  +  0‘dj)  >  0  for  j  £  0*  it  follows  for  some  new  range  0t+1  <  0  <  0‘ 
that  Cj(0)  >  0.  We  are  now  ready  to  repeat  the  iterative  process. 

Convergence  is  guaranteed  in  a  finite  number  of  iterations  because  repetition  of  a 
canonical  form  for  some  iteration  t  +  t  with  a  0  <  0t+1  would  imply  a  lowering  of  0  below 
the  calculated  minimum  0t+l,  a  contradiction.  On  all  iterations,  the  incoming  column  has 
c,  <  0  so  that  if  there  is  no  stalling,  there  will  be  a  positive  decrease  in  z.  On  all  iterations 
0  is  strictly  decreasing. 


3.  Updating 

First  way:  If  vectors  c  and  d  of  iteration  t  —  1  are  stored,  then  those  of  iteration  t  can 
be  computed  by  a  single  solve  for  “prices”  and  a  single  “pricing  out ”  of  columns.  For 
pricing-out  vector  p,  the  r-th  row  of  the  inverse  of  J5,_l  is  used.  This  requires  one  to  solve 


pB *  1  =  Ur  where  Ur  is  unit  vector  r.  Columns  axe  priced  out  by  computing  arJ  =  pAy. 
The  update  formulas  are 

updated  c,  =  c}  —  Aor}  where  A  =  c,/ar, 
updated  dj  =  dj  —  parj  where  p  —  d,fart. 


Second  way:  Instead,  the  updating  of  cy  can  be  done  in  the  usual  way  by  determining 
7r  by  a  single  solve  =  cp  where  cp  —  (cy, ,  ci2 , . . . ,  eym )  and  (3  —  0*.  This  way  also 
requires  a  single  pricing  out, 

updated  cy  =  cy  —  7r‘Ay 

updated  dj  =  dj  —  (p/X)(cj  -  updated  cy) 

which  can  easily  be  verified  by  eliminating  ary  from  the  first  way  of  updating.  This  way 
requires  not  overlaying  updated  cy  on  stored  cy  until  after  updated  dj  is  computed. 

Third  way:  Edward  Klotz  (private  communication)  recommends  a  double  pricing 
scheme  that  requries  one  solve  and  no  storing  of  c  and  d, 

updated  Cy  =  Cy  —  7T*  Ay 
updated  dj  =  dj  —  otA.j 

where  a *  is  generated  without  a  solve  by 

updated  o  =  o  +  {d,/c,)  (updated  n  —  n)  , 

which  requires  only  a  temporary  storing  of  components  7r»  of  n  as  it  is  computed  to  compute 
the  corresponding  components  of  <t‘.  The  above  formula  for  updated  o  =  o*  is  obtained 
by  eliminating  p  from  the  relations: 

7rf  =  tt4-1  4-  ( c_./ar,)p 
o'  =  c*-1  +  ( d,/ar,)p  . 


4.  Benefits 


1.  As  noted  earlier  )|Ay||  can  be  any  positive  number.  If  it  is  a  column  norm  of  some 
kind,  however,  then  the  particular  choice  of  initial  dj  =  ||Ay||(l  4-  £y)  makes  the  selection 
of  s,  the  incoming  column  independent  of  the  unit  for  measuring  the  j-th  activity.  The 
random  choice  of  ey  avoids  dual  degeneracy  with  probability  1.  The  choice  of  norm  can  be 
the  Euclidean  norm  ||  Ay||2,  or  the  sum  of  positive  components  of  Ay  or  max*  Ay  providing 
some  Ay  >  0,  or  max,  j  Ay  |.  Any  of  these  will  render  column  selection  unit  free. 

2.  The  choice  of  incoming  column  s  by  the  parameterization  scheme  appears  to  be  at 
least  as  good  a  choice  as  the  usual  rule  s  =  argmin  cy. 
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3.  Near  dual  degeneracy  does  not  appear  likely  to  cause  the  same  “treading  of  water 
and  getting  nowhere”  as  does  degeneracy  in  the  primal  because  all  near  tieing  columns  are 
promising  columns  to  enter  the  basis  and  should  be  considered. 

4.  Decreasing  dual  infeasibility  during  the  iterations  when  decrease  in  objective  in  z  is 
stalled  seems  to  be  a  better  strategy  than  the  artificial  schemes  like  the  lexico  min  ratio, 
Bland’s  rule,  Wolfe’s  rule,  or  the  random  rule  that  have  been  proposed  to  avoid  cycling. 


5.  Drawbacks 

1.  When  partial  pricing  is  used,  it  is  recommended  that  a  parametric  scheme  be  imposed 
on  each  partition  separately  with  6  replaced  with  separate  $lt  02, . . .  for  each  partition.  The 
reduction  of  0,  in  one  partition  could  cause  the  value  of  the  0,+1  of  the  next  partition  to  be 
reinitiated  at  a  higher  value  than  the  last  reduction  of  0f+i  and  therefore  it  is  doubtful  that 
one  can  prove  convergence  when  the  primal  is  stalled.  Convergence  for  unstalled  steps  is, 
of  course,  guaranteed. 

2.  If  the  first  or  second  way  is  used  to  do  the  updating,  the  cost  per  iteration  requires 
only  four  more  instructions  per  column  to  maintain  row  vectors  c  and  d  plus  the  cost  of 
initially  storing  two  starting  vectors  c  =  c  and  d  =  d  so  there  should  be  only  an  insignificant 
increase  in  CPU  time. 


6.  Tests  on  Practical  Problems 

The  Systems  Optimization  Laboratory  of  Stanford  University,  Operations  Research  De¬ 
partment,  has  collected  a  number  of  test  problems  drawn  from  practical  sources.  These 
are  used  in  systematic  trials  comparing  various  proposed  techniques  for  solving  linear  pro¬ 
grams.  See  for  example  Irvin  Lustig  [4j.  As  part  of  his  forthcoming  Ph.D.  thesis,  Edward 
Klotz  tested  a  number  of  proposed  methods,  including  some  of  his  own  invention  for  re¬ 
ducing  the  number  of  iterations  and  the  CPU  time.  Among  them  is  the  method  proposed 
in  this  paper  [3]. 

All  the  experiments  were  run  by  Klotz  on  a  DEC  MicroVax  using  MINOS  5.1  on  nine 
test  problems  specially  selected  because  a  high  percentage  of  their  iterations  were  blocked 
using  the  “regular”  simplex  method.  The  proposed  method  was  compared  with  the  latter. 
Neither  method  used  the  scaling  or  partial  pricing  options  of  MINOS.  Except  for  column 
selection,  all  features  of  MINOS  were  identical  for  both  methods.  See  Table  comparing 
iterative  count.  The  CPU  time  shown  for  the  parametric  method  may  be  misleading 
because  none  of  the  ways  recommended  earlier  for  doing  the  updating  were  used.  Instead 
a  slower  (but  easier  to  implement)  double-solve  double-pricing  scheme  was  used.  A  better 
implementation  is  expected  to  show  CPU  time  per  iteration  about  equal  to  the  iteration 
time  per  iteration  of  the  regular  simplex.  If  so,  then  the  iteration  ratio  shown  for  the 
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test  problems  would  be  a  better  measure  for  comparing  the  relative  efficiency  of  the  two 
methods  for  very  degenerate  problems. 

Comparative  Iteration  Count  of  Regular  vs.  Parametric 
Simplex  Method  on  Nine  Highly  Degenerate  Problems 


Problem  Size 


PROBLEM 


NAME 

#  ROWS 

#COLS.* 

KB2 

46 

41 

DEGENl 

67 

72 

TUFF 

371 

587 

DEGEN2 

445 

534 

WOODIP 

486 

2594 

NZFRI 

624 

3521 

WOODW 

1089 

8405 

DEGEN3 

1504 

1818 

CYCLE 

2234 

2857 

*  excludes  slacks 

Total  Iterations 

%  Blocked 

Iterations 

“REGULAR”  PARAMETRIC 
SIMPLEX  SCHEME 


Parametric  Iterations/Regular  Iterations 
Parametric  CPU  /  Regular  CPU  =  .60 
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Abstract 


All  the  standard  methods  for  avoiding  cycling  in  the  simplex  algorithm 
set  up  a  subprogram  related  to  the  blocking  rows  in  the  canonical  form  and 
look  for  an  improving  homogeneous  solution.  The  method  proposed  in  this 
paper  does  not.  Instead,  the  Gass-Saaty  parametric  method  applied  to  the 
objective  (cost)  form  is  used  to  choose  the  incoming  column  [2].  The 

pivot-row  choice  among  blocking  rows,  i.e.,  those  tied  for  pivot,  is 

arbitrary.  A  simple  anti-cycling  device  is  used  which  avoids  dual  degeneracy 
of  the  parameterized  objective  with  probability  one.  Tests  were  run  on  nine 
highly  degenerate  practical  test  problems  ranging  in  size  from  small  to 
large.  Using  MINOS  software  [5],  the  standard  simplex  method  required  31,195 
iterations  of  which  20,504  (or  66%)  were  blocked  pivots.  Using  the 
parametric  scheme  required  14,379  iterations,  of  which  9,481  of  66%  were  also 
blocked.  The  reduction  in  the  number  of  iterations  for  this  set  of  highly 
degenerate  test  problems  is  iver  50%.  CPU  time  reduction  under  the  current 
way  of  implementation  is  40%. 
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